The Rôle of a priori Biases in Unsupervised Learning of Visual Representations: A Robotics Experiment

An infant’s learning of visual representations is entirely unsupervised. While unsupervised neural network learning architectures had some successes in predicting the receptive field properties of early visual representations in the brain, it remains unclear how the formation of higher level representations can be understood. This paper argues that in order to understand the formation of these higher level representations we must take the active and purposive nature of biological vision into account. Biological organisms can actively shape the statistics of what they see and what they learn according to a priori biases or to meet particular needs. To study the effects of a priori biases on unsupervised learning we use an autonomous robot whose learning is focused on “relevant” stimuli. In a first experiment we establish that if the robot restricts learning to “interesting” image regions, the results differ dramatically from learning on random image patches. In particular, if learning focuses on image regions showing motion and skin color, the robot spontaneously develops units that can be described as face detectors. In a second experiment we show how the exploitation of temporal continuity allows the robot to generalize its innate knowledge of what stimuli are relevant to new contexts. In particular, the robot develops color units that describe the color of faces under illumination conditions that are different from the one for which the a priori bias was designed.